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Abstract. In the recent progress |BE1| . [Me| and [Z2], the well- 
known JC (Jacobian conjecture) ( [BCW| . [E]) has been reduced 
to a VC (vanishing conjecture) on the Laplace operators and HN 
(Hessian nilpotent) polynomials (the polynomials whose Hessian 
matrix are nilpotent). In this paper, we first show that the vanish- 
ing conjecture above, hence also the JC, is equivalent to a vanishing 
conjecture for all 2nd order homogeneous differential operators A 
and A-nilpotent polynomials P (the polynomials P{z) satisfying 
^mpm _ Q £gj, all rn > 1). We then transform some results in the 
literature on the JC, HN polynomials and the VC of the Laplace 
operators to certain results on A-nilpotent polynomials and the 
associated VC for 2nd order homogeneous differential operators 
A. This part of the paper can also be read as a short survey 
on HN polynomials and the associated VC in the more general 
setting. Finally, we discuss a still-to-be-understood connection of 
A-nilpotent polynomials in general with the classical orthogonal 
polynomials in one or more variables. This connection provides 
a conceptual understanding for the isotropic properties of homo- 
geneous A-nilpotent polynomials for 2nd order homogeneous full 
rank differential operators A with constant coefficients. 



1. Introduction 

Let z = {zi, Z2, ■ . ■ , Zn, . . .) be a sequence of free commutative vari- 
ables and D = {Di, D2, . . . , Dn, ■ ■ ■) with Di ■= ^ {i > !)■ For any 
n > 1, denote by An (resp. An) the algebra of polynomials (resp. formal 
power series) in Zi [1 < i < n). Furthermore, we denote by 2)[^n] or 
D[n] (resp.D[yi„] or D[n]) the algebra of differential operators of the 
polynomial algebra An (resp. with constant coefficients). Note that, for 
any k > n, elements of D[n] are also differential operators of A^ and 
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Ak- For any d > 0, denote by Dd[n] the set of homogeneous differen- 
tial operators of order d with constants coefficients. We let A (resp. 71) 
be the union of An (resp.yi„) {n > 1), D (resp. D) the union of D[n] 
(resp. D[n]) {n > 1), and, for any d > 1, the union of 2)rf[n] (n > 1). 

Recall that JC (the Jacobian conjecture) which was first proposed 
by Keller |Ke] in 1939, claims that, for any polynomial map F of 
with Jacobian j{F) = 1, its formal inverse map G must also he a 
polynomial map. Despite intense study from mathematicians in more 
than sixty years, the conjecture is still open even for the case n = 2. 
For more history and known results before 2000 on JC, see |BCW] . jE] 
and references there. 

Based on the remarkable symmetric reduction achieved in |BElj . 
[Me] and the classical celebrated homogeneous reduction [BCWj and 
[Y] on JC, the author in \L2\ reduced JC further to the following 
vanishing conjecture on the Laplace operators A„ := Yl^=i-^'i the 
polynomial algebra 7l„ and HN {Hessian nilpotent) polynomials P{z) G 
An, where we say a polynomial or formal power series P{z) G An is 
HN if its Hessian matrix Hes (P) := {-^-§^)nxn is nilpotent. 

Conjecture 1.1. For any HN {homogeneous) polynomial P{z) G An 
{of degree d = 4), we have /\^P"^^^{z) = when m >> 0. 

Note that, the following criteria of Hessian nilpotency were also 
proved in Theorem 4.3, |Z2j . 

Theorem 1.2. For any P{z) G An with o{P{z)) > 2, the following 
statements are equivalent. 

(1) P{z) IS HN. 

(2) A™p™ = for any m>l. 

(3) A™P™ = Oforanyl<m< n. 

Through the criteria in the proposition above. Conjecture 11.11 can 
be generalized to other differential operators as follows (see Conjecture 
O below). 

First let us fix the following notion that will be used throughout the 
paper. 

Definition 1.3. Let A G Ti[An] and P{z) G An. We say P{z) is 
A-nilpotent z/ A^P™ = for any m > 1. 

Note that, when A is the Laplace operator A^^, by Theorem II. 2[ a 
polynomial or formal power series P{z) G An is A-nilpotent iff it is HN. 

With the notion above. Conjecture 11.11 has the following natural 
generalization to differential operators with constant coefficients. 
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Conjecture 1.4. For any n >1 and A G ©[n], if P(z) G An is A- 
nilpotent, then A™'P'"+^ = when m » 0. 

We call the conjecture above the vanishing conjecture for differential 
operators with constant coefficients and denote it by VC. The special 
case of VC with P{z) homogeneous is called the homogeneous vanish- 
ing conjecture and denoted by HVC. When the number n of variables 
is fixed, VC (resp. HVC) is called (resp. homogeneous) vanishing con- 
jecture in n variables and denoted by VC[n] (resp. HVC[n]). 

Two remarks on VC follows. First, due to a counter-example 

given by M. de Bondt (see example 12.41) . VC does not hold in general 
for differential operators with non-constant coefficients. Secondly, one 
may also allow P{z) in VC to be any A-nilpotent formal power series. 
No counter-example to this more general VC is known yet. 

In this paper, we first apply certain linear automorphisms and Lef- 
schetz's principle to show Conjecture 11.11 hence also JC, is equivalent 
to VC or HVC for all 2nd order homogeneous differential operators 
A G D2 (see Theorem 12. 91) . We then in Section [3] transform some results 
on JC, HN polynomials and Conjecture 11.11 obtained in [Waj . |BE2j . 
|BE3] . |Z2] . |Z3] and |EZ] to certain results on A-nilpotent (A G D2) 
polynomials and VC for A. Another purpose of this section is to give 
a survey on recent study on Conjecture [TTT] and HN polynomials in the 
more general setting of A G D2 and A-nilpotent polynomials. This is 
also why some results in the general setting, even though their proofs 
are straightforward, are also included here. 

Even though, due to M. de Bondt's counter-example (see Example 
12.41) . VC does not hold for all differential operators with non-constant 
coefficients, it is still interesting to consider whether or not VC holds 
for higher order differential operators with constant coefficients; and 
if it also holds even for certain families of differential operators with 
non-constant coefficients. For example, when A = D'^ with a G N" and 
> 2, VC[?T,] for A is equivalent to a conjecture on Laurent polynomi- 
als (see Conjecture 13. 2 II) . This conjecture is very similar to a non-trivial 
theorem (see Theorem I3.2UI) on Laurent polynomials, which was first 
conjectured by O. Mathieu [Ma] and later proved by J. Duistermaat 
and W. van der Kallen [DKj . 

In general, to consider the questions above, one certainly needs to get 
better understandings on the A-nilpotency condition, i.e. A"^P"^ = 
for any m > 1. One natural way to look at this condition is to consider 
the sequences of the form |A™P™ | m > 1} for general differential op- 
erators A and polynomials P{z) G A. What special properties do these 
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sequences have so that VC wants them all vanish? Do they play any 
important roles in other areas of mathematics? 

The answer to the first question above is still not clear. The answer 
to the later seems to be "No". It seems that the sequences of the form 
l^mpm I ^ y appear very often in mathematics. But the 

answer turns out to be "Yes" if one considers the question in the setting 
of some localizations S of An- Actually, as we will discuss in some detail 
in subsection 14.11 all classical orthogonal polynomials in one variable 
have the form |A'"P'^ | m > 1} except there one often chooses P{z) 
from some localizations S of An and A a differential operators of B. 
Some classical polynomials in several variables can also be obtained 
from sequences of the form {A^P'" | m > 1} by a slightly modified 
procedure. 

Note that, due to their applications in many different areas of math- 
ematics, especially in ODE, PDE, the eigenfunction problems and rep- 
resentation theory, orthogonal polynomials have been under intense 
study by mathematicians in the last two centuries. For example, in 
[SHW] published in 1940, about 2000 published articles mostly on one- 
variable orthogonal polynomials have been included. The classical ref- 
erence for one- variable orthogonal polynomials is [Si] (see also [AS], 
[C] . [Si]). For multi- variable orthogonal polynomials, see |DXj . |Ko] 
and references there. 

It is hard to believe that the connection discussed above between 
A-nilpotent polynomials or formal power series and classical orthog- 
onal polynomials is just a coincidence. But a precise understanding 
of this connection still remains mysterious. What is clear is that, A- 
nilpotent polynomials or formal power series and the polynomials or 
formal power series P{z) G An such that the sequence {A'^P™' | m > 1} 
for some differential operator A provides a sequence of orthogonal poly- 
nomials lie in two opposite extreme sides, since, from the same sequence 
l^mpm I J77, > 1}^ the former provides nothing but zero; while the later 
provides an orthogonal basis for An- 

Therefore, one naturally expects that A-nilpotent polynomials P{z)e 
An should be isotropic with respect to a certain C-bilinear form of An- 
It turns out that, as we will show in Theorem 14. 101 and Corollary 14. 11^ 
it is indeed the case when P{z) is homogeneous and A G is of 

full rank. Actually, in this case A^P™"*"^ {m > 0) are all isotropic with 
respect to same properly defined C-bilinear form. Note that. Theorem 
14. 101 and Corollary 14.111 are just transformations of the isotropic prop- 
erties of HN nilpotent polynomials, which were first proved in |Z2j. 
But the proof in |Z2] is very technical and lacks any convincing inter- 
pretations. From the "formal" connection of A-nilpotent polynomials 
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and orthogonal polynomials discussed above, the isotropic properties 
of homogeneous A-nilpotent polynomials with A e D2[n] of full rank 
become much more natural. 

The arrangement of the paper is as follows. In Section [21 we mainly 
show that Conjecture II. H hence also JC, is equivalent to VC or HVC 
for all A G ©2 (see Theorem 12.91) . One consequence of this equivalence 
is that, to prove or disprove VC or JC, the Laplace operators are not 
the only choices, even though they are the best in many situations. 
Instead, one can choose any sequence A„^ G D2 with strictly increasing 
ranks (see Proposition 12. 101) . For example, one can choose the "Laplace 
operators" with respect to the Minkowski metric or symplectic metric, 
or simply choose A to be the complex 9-Laplace operator Aq ^ (A; > 1) 
in Eq. dflU). 

In Section [31 we transform some results on JC, HN polynomials 
and Conjecture 11.11 in the literature to certain results on A-nilpotent 
(A G ©2) polynomials P{z) and VC for A. In subsection 13.11 we 
discuss some results on the polynomial maps and PDEs associated 
with A-nilpotent polynomials for A G ©2 [n] of full rank (see Theorems 
I3.1H3.3I) . The results in this subsection are transformations of those 
in [Zlj and |Z2j on HN polynomials and their associated symmetric 
polynomial maps. 

In subsection 13.21 we give four criteria of A-nilpotency (A G ©2) (see 
Propositions 13. 4[ 13.61 13.71 and l3.1Up . The criteria in this subsection 
are transformations of the criteria of Hessian nilpotency derived in 
|Z2] and [Z3]. In subsection 13.31 we transform some results in |BCW] . 
[Waj and ^ on JC; |BE2] and |BE3] on symmetric polynomial maps; 
|Z2] . |Z3] and [EZ] on HN polynomials to certain results on VC for 
A G D2. Finally, we recall a result in |Z3] which says, VC over fields 
k of characteristic p > 0, even under some conditions weaker than A- 
nilpotency, actually holds for any differential operators A of k[z] (see 
Proposition 13.221 and Corollary 13.231) . 

In subsection 13.41 we consider VC for high order differential opera- 
tors with constant coefficients. In particular, we show in Proposition 
13.181 VC holds for A = 6^ {k > 1), where 5 is a derivation of A. In 
particular, VC holds for any A G Di (see Corollary I3.19p . We also 
show that, when A = D'^ with a G N" and |a| > 2, VC is equivalent 
to a conjecture. Conjecture 13.211 on Laurent polynomials. This con- 
jecture is very similar to a non-trivial theorem (see Theorem 13.201) first 
conjectured by O. Mathieu [Ma] and later proved by J. Duistermaat 
and W. van der Kallen [DKj . 
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In subsection 14.11 by using Rodrigues' formulas Eq. fl4.ip . we show 
that all classical orthogonal polynomials in one variable have the form 
l^mpm I ^ > X j ^Qj- some P{z) in certain localizations S of An and A a 
differential operators of S. We also show that some classical polynomi- 
als in several variables can also be obtained from sequences of the form 
l^mpm I ^ > X} with a slight modification. Some of the most classical 
orthogonal polynomials in one or more variables are briefly discussed 
in Examples I4.2H4.5[ 14.81 and 14. 9[ In subsection 14.21 we transform the 
isotropic properties of homogeneous HN homogeneous polynomials de- 
rived in [Z2j to homogeneous A-nilpotent polynomials for A G ©2 [n] of 
full rank (see Theorem I4.1UI and Corollary 14. lip . 

Acknowledgment: The author is very grateful to Michiel de Bondt 
for sharing his counterexample (see Example 12. 4p with the author, and 
to Arno van den Essen for inspiring personal communications. The 
author would also like to thank the referee very much for many valuable 
suggestions to improve the first version of the paper. 

2. The Vanishing Conjecture for the 2nd Order 
Homogeneous Differential Operators with Constant 

Coefficients 

In this section, we apply certain linear automorphisms and Lef- 
schetz's principle to show Conjecture 11.11 hence also JC, is equivalent 
to VC or HVC for all A G D2 (see Theorem 12. 9p . In subsection 12. H we 
fix some notation and recall some lemmas that will be needed through- 
out this paper. In subsection 12.21 we prove the main results of this 
section. Theorem 12.91 and Proposition 12.101 

2.1. Notation and Preliminaries. Throughout this paper, unless 
stated otherwise, we will keep using the notations and terminology in- 
troduced in the previous section and also the ones fixed as below. 

(1) For any P{z) G An, we denote by VP the gradient of P{z), i.e. 



(2) For any n > 1, we let SM{n,C) {resp. SGL{n,C)) denote the 
symmetric complex n x n (resp. invertible) matrices. 

(3) For any A = (0^) G SM{n, C), we set 



we set 



(2.1) 



VP{z):= {DiP, D2P,...,DnP). 



n 



(2.2) 
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Note that, for any A G 1152 there exists a unique A G 
SM{n, C) such that A = A^. We define the rank of A = 
simply to be the rank of the matrix A. 

(4) For any n > 1, A G D2[n] is said to be full rank if A has rank n. 
The set of full rank elements of D2H will be denoted by 1152 [^]- 

(5) For any r > 1, we set 

r 

(2.3) K:=Y.Dl 

i=l 

Note that A^ is a full rank element in ©2 [f] but not in D2 [n] for 
any n > r. 



For U G GL{n,C), we define 

(2.4) ^u-An^ An 

P{z) P{U-'^z) 

and 

(2.5) ^[/:2)N ^ T)[n] 

A ^ $[/ o A o 

It is easy to see that, (resp. '^u) is an algebra automorphism of 
An (resp. 2) [n]). Moreover, the following standard facts are also easy 
to check directly. 

Lemma 2.1. (a) For any U = {uij) G GL{n,C), P{z) G An and 
A G ©[ra], we have 

(2.6) $t;(AP) = ^^(A)$t;(P). 

{h) For any 1 < i < n and f{z) G An we have 

n 

^c/(/P)) = /(f/^^). 
/n particular, for any A G SM{n, C), we /lawe 

(2.7) *c/(Aa) = At;^f;.. 

The following lemma will play a crucial role in our later arguments. 
Actually the lemma can be stated in a stronger form (see [Huj . for 
example) which we do not need here. 
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Lemma 2.2. For any A G SM{n, C) of rank r > 0, there exists U G 
GL{n, C) such that 



Combining Lemmas l2.ll and f2.2l it is easy to see we have the following 
corollary. 

Corollary 2.3. For any n > 1 and A, S G ©2M of same rank, there 
exists U G GL{n, C) such that ^'[/(A) = H. 

2.2. The Vanishing Conjecture for the 2nd Order Homoge- 
neous Differential Operators with Constant Coefficients. In 

this subsection, we show that Conjecture II. ![ hence also JC, is actu- 
ally equivalent to VC or HVC for all 2nd order homogeneous differ- 
ential operators A G D2 (see Theorem 12.91) . We also show that the 
Laplace operators are not the only choices in the study of VC or JC 
(see Proposition 12. 101 and Example 12.111) . 

First, let us point out that VC fails badly for differential opera- 
tors with non-constant coefficients. The following counter-example was 
given by M. de Bondt [Bj. 

Example 2.4. Let x be a free variable and A = x-^. Let P{x) = x. 
Then one can check inductively that P{x) is A-nilpotent, but \'^p^+^ ^ 
for any m > 1. 

Lemma_2.5. For any A G D[n], U G GL{n,C), ^ e SM{n,C) and 
P{z) G An, we have 

(a) P{z) is A-nilpotent iff' ^u{P) is '^u{^)-'nilpotent. In particular, 
P{z) is IS. A-nilpotent iff^u{P) = P{U~^z) is Auau^ -nilpotent. 

(b) VC[n] {resp. HVC[n]) holds for A iff it holds for "^ui^)- In 
particular, VC[n] (resp. HVC [n]) holds for iff it holds for 



(2.8) 




Proof: Note first that, for any m, > 1, we have 



= [^uiATi^uPf. 



When A = A^, by Eq. (12.71) . we further have 



{A^P')=A^AuA^uP)'. 
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Since ${/ (resp. is an automorphism of An (resp. I'fn]), it is easy 
to check directly that both (a) and (6) follow from the equations above. 
□ 

Combining the lemma above with Corollary I2.3[ we immediately 
have the following corollary. 

Corollary 2.6. Suppose lIVC[n] {resp.\C[n]) holds for a differential 
operator A G D2[ri] of rank r > 1. Then HVC[n] {resp.\C[n]) holds 
for all differential operators S G of rank r. 

Actually we can derive much more (as follows) from the conditions 
in the corollary above. 

Proposition 2.7. (a) Suppose lIVC[n] holds for a full rank A G ID'2[r;.]. 
Then, for any k <n, HVC[/c] holds for all full rank S G D2[/i;]. 

(6) Suppose VC[n] holds for a full rank A G II52M- Then, for any 
m > n, VC[m] holds for all S G ©2['^] of rank n. 

Proof: Note first that, the cases k = n m (a) and m = n in (6) 
follow directly from Corollary 12.61 So we may assume k < n'm (a) and 
m > n in (6). Secondly, by Corollary 12. 6[ it will be enough to show 
HVC[A;] {k < n) holds for for (a) and VC[m] (m > n) holds for 
An for (6). 

(a) Let P G Ak a homogeneous A^-nilpotent polynomial. We view 
Afc and P as elements of ©2!^] and >A„, respectively. Since P does not 
depend on (A; + 1 < i < n), for any m, £ > 0, we have 

A^P^ = A^P^. 

Hence, P is also A„-nilpotent. Since HVC[n] holds for A„ (as pointed 
out at the beginning of the proof), we have A^P'"^^ = A^P'^^^ = 
when m >> 0. Therefore, HVC[/c] holds for A^. 

{h) Let K be the rational function field C(z„+i, . . . , Zm)- We view Am 
as a subalgebra of the polynomial algebra K[zi^ . . . , 2;„] in the standard 
way. Note that the differential operator A„ = Ym=i extends 
canonically to a differential operator of [2:1, ... , z^] with constant co- 
efficients. 

Since VC[n] holds for A„ over the complex field (as pointed out 
at the beginning of the proof), by Lefschetz's principle, we know that 
VC[n] also holds for A„ over the field K. Therefore, for any A„- 
nilpotent P{z) G Am, by viewing A„ as an element of32{K[zi, . . . , z„]) 
and P{z) an element of K[zi, . . . , Zn] (which is still A„-nilpotent in the 
new setting), we have A^P'^"'"^ = when k >> 0. Hence VC[m] holds 
for P(2) G /1„^ and A„ G D2[m]. □ 
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Proposition 2.8. Suppose HVC[n] holds for a differential operator 
A G D2[Ti] with rank r < n. Then, for any k > r, VC[/i;] holds for all 
S G D2[^] of rank r. 

Proof: First, by Corollary 12.61 we know HVC[n] holds for A^. To 
show Proposition [231 by Proposition 12. 71 (6), it will be enough to show 
that VC[r] holds for A,.. 

Let P & Ar (Z An be a A^-nilpotent polynomial. If P is homoge- 
neous, there is nothing to prove since, as pointed out above, HVC[n] 
holds for A,.. Otherwise, we homogenize P{z) to P E Ar+i C An- 
Since A^. is a homogeneous differential operator, it is easy to see that, 
for any m,k > 1, A;"P*-' = iff A;"P*^ = 0. Therefore, P G /l„ is 
also A^-nilpotent when we view A^ as a differential operator of An- 
Since HVC[n] holds for A^, we have that A^P"'+^ = when m >> 0. 
Then, by the observation above again, we also have A™P™+^ = when 
m » 0- Therefore, VC[r] holds for A^. □ 

Now we are ready to prove our main result of this section. 

Theorem 2.9. The following statements are equivalent to each other. 

(1) JC holds. 

(2) HVC[n] {n > 1) hold for the Laplace operator A„. 

(3) VC[n] (n > 1) hold for the Laplace operator A„. 

(4) HVCM (n > 1) hold for all A G DzM- 

(5) YC[n] {n > 1) hold for all A G DzM- 

Proof: First, the equivalences of (1), (2) and (3) have been estab- 
lished in Theorem 7.2 in p]. While (4) (2), (5) (3) and (5) (4) 
are trivial. Therefore, it will be enough to show (3) =^ (5). 

To show (3) =^ (5), we fix any n > 1. By Corollary 12.61 it will 
be enough to show VC[n] holds for A^ (1 < r < ra). But under the 
assumption of (3) (with = r), we know that VC[r] holds for A^. 
Then, by Proposition 12.71 (6), we know VC[n] also holds for A^. □ 

Next, we show that, to study HVC, equivalently VC or JC, the 
Laplace operators are not the only choices, even though they are the 
best in many situations. 

Proposition 2.10. Let {uk \ k > 1} be a strictly increasing sequence of 
positive integers and {A„^ \ k > 1} a sequence of differential operators 
in D2 with rank{An^) = Uk {k > 1). Suppose that, for any k > 1, 
HVC[A^fc] holds for An,, for some Nk >nk- Then, the equivalent state- 
ments in Theorem \2.9[ hold. 
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Proof: We show, under the assumption in the proposition, the state- 
ment (2) in Theorem 12.91 holds, i.e. for any n > 1, HVC[n] {n > 1) 
holds for the Laplace operator A„ G 

For any fixed n > 1, let A; > 1 such that > n. If Nj. = n^, 
then, by Proposition 12.71 (a), we have HVCfri] (n > 1) holds for the 
Laplace operator A„ G If Nk > Uk, then, by Proposition 12.81 we 

know VC[nfc] (hence also HVC[nfc]) holds for A„^. Since rik > n, by 
Proposition 12. 7[ (a) again, we know HVC[ri] does hold for the Laplace 
operator A„. □ 

Example 2.11. Besides the Laplace operators, by Proposition \2A^ the 
following sequences of differential operators are also among the most 
natural choices. 

(1) Let Uk = k {k > 2) {or any other strictly increasing sequence of 
positive integers) . Let A^, be the "Laplace operator" with respect 
to the standard Minkowski metric o/M"*^. Namely, choose 

k 

(2.9) A, = Dl-Y,Dl 

1=2 

(2) Choose Hk = 2k {k > 1) (or any other strictly increasing se- 
quence of positive even numbers) . Let be the ''Laplace op- 
erator" with respect to the standard symplectic metric on , 
i.e. choose 

k 

(2.10) A2fc = ^AA+fc. 

1=1 

(3) We may also choose the complex Laplace operators Aq instead of 
the real Laplace operator IS.. More precisely, we choose Uk = 2k 
for any k > 1 and view the polynomial algebra of Wi {I < i < 
2k) over C as the polynomial algebra C[zi,Zi \ 1 < i < k] by 
setting Zi = Wi + Wi^k for any 1 < i < k. Then, for any 
k > 1, we set 

i=l 

(4) More generally, we may also choose = ^An^ , where G N 
and An^ G SM{nk,'C) {not necessarily invertible) {k > 1) with 
strictly increasing ranks. 
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3. Some Properties of A^-Nilpotent Polynomials 

As pointed earlier in Section [T] (see page [2]), for the Laplace operators 
> 1), the notion A„-nilpotency coincides with the notion of Hes- 
sian nilpotency. HN (Hessian nilpotent) polynomials or formal power 
series, their associated symmetric polynomial maps and Conjecture ll.il 
have been studied in |BE2] . |BE3] . |Z1] |Z3] and ( EZj . In this section, 
we apply Corollary 12.31 Lemma 12.51 and also Lefschetz's principle to 
transform some results obtained in the references above to certain re- 
sults on A-nilpotent (A G ©2) polynomials or formal power series, VC 
for A and also associated polynomial maps. Another purpose of this 
section is to give a short survey on some results on HN polynomials and 
Conjecture 11.11 in the more general setting of A-nilpotent polynomials 
and VC for differential operators A G D2. 

In subsection 13. H we transform some results in [Zl] and jZ2j to the 
setting of A-nilpotent polynomials for A G ©2 [n] of full rank (see Theo- 
rems [3lTH3l3])- In subsection l3.2[ we derive four criteria for A-nilpotency 
(A G D2) (see Propositions 13.41 13.61 13.71 and l3.10p . The criteria in this 
subsection are transformations of the criteria of Hessian nilpotency de- 



rived in [Z2] and [Zo . 

In subsection 13.31 we transform some results in [BCW] , |Wa] and 
[Y] on JC; |BE2] and |BE3] on symmetric polynomial maps; jZ2] . 
and |EZj on HN polynomials to certain results on VC for A G 



In subsection 13.41 we consider VC for high order differential operators 
with constant coefficients. We mainly focus on the differential operators 
A = D'^ (a G N"). Surprisingly, VC for these operators is equivalent 
to a conjecture (see Conjecture I3.2ip on Laurent polynomials, which 
is similar to a non-trivial theorem (see Theorem 13. 20p first conjectured 
by O. Mathieu [Maj and later proved by J. Duistermaat and W. van 
der Kallen [DK] . 



3.1. Associated Polynomial Maps and PDEs. Once and for all in 

this section, we fix any n > 1 and A G SM{n, C) of rank 1 < r < n. We 
use z and D, unlike we did before, to denote the n-tuples (zi, Z2, ■ ■ ■ , Zn) 
and (Di, L'2, • • • , respectively. We define a C-bilinear form (-, ■)a 
by setting {u,v)a '■= Av for any u,v E C". Note that, when A = 
Inxn, the bilinear form defined above is just the standard C-bilinear 
form of C", which we also denote by (■,■). 

By Lemma [2.21 we may write A as in Eq. (12.80 . For any P{z) G An, 
we set 



(3.1) 



P{z) = <^u^Piz) = P{Uz). 
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Note that, by Lemma \2.1\ (6), we have \E'j^^(A^) = A^. By Lemma 
12.51 (a), P{z) is A^-nilpotent iff P{z) is Ar-nilpotent. 

Theorem 3.1. Let t be a central parameter. For any P{z) G An with 
o{P{z)) >2andAe SGL{n, C), set FA,t{z) := z - tAVP{z). Then 

(a) there exists a unique QA,t{,z) G C[t][[z]] such that the formal 
inverse map GA,t{z) of FA,t{z) is given by 

(3.2) GaA^) =^ + tA^QAAz)■ 

{b) The QA,t{^) G in (a) is the unique formal power series 

solution of the following Cauchy problem: 



(3.3) 



■^(z) = i(vgA„vgA,t)A, 

QA,t=oi^) = P{z). 



Proof: Let P as given in Eq. (13. ip and set 

(3.4) FA,t{z)=z-tVP{z). 

By Theorem 3.6 in [Zlj, we know the formal inverse map GA,t{^) of 
FA,t{z) is given by 

(3.5) GA,t{z)=z + t^QAAz). 

where QaA"^) G C[t][[z]] is the unique formal power series solution of 
the following Cauchy problem: 



(3.6) 



^-^{z) = \{yQA,uVQA,). 

QA,t=oiz) = Piz). 



From the fact that VP{z) = {U"^'VP){Uz), it is easy to check that 

(3.7) i^u o FA,t o $j}^)(^) =z- tAWP{z) = FaA^). 
which is the formal inverse map of 

(3.8) ($^7 O GA,t o <l>u'){z) = z + tiJJVQA^tW'^z). 
Set 

(3.9) QaA^)-=QaAU'^A- 
Then we have 

VQaA^) = {U^)-\VQA,tAU~'z), 

(3.10) U^VQaA^) = {VQA,tAU-'z), 
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Multiplying U to the both sides of the equation above and noticing 
that A = Ulf^ by Eq. (12.81) since A is of full rank, we get 

(3.11) AVQaA^) = {UVQA,t){U-'z). 

Then, combining Eq. (13. 8 P and the equation above, we see the formal 
inverse GA,t{z) of FA,t{z) is given by 

(3.12) GaA^) = i^^u o GA,t o %'){z) = z + IAVQaAA- 

Applying to Eq. (ESI) and by Eqs. ([331), fl3J0D . we see that QaAA is 
the unique formal power series solution of the Cauchy problem Eq. (13. 3p . 
□ 

By applying the linear automorphism $(/ of C[[2;]] and employing a 
similar argument as in the proof of Theorem 13.11 above, we can gen- 
eralize Theorems 3.1 and 3.4 in |Z2] to the following theorem on A^- 
nilpotent {A G SGL{n,C)) formal power series. 

Theorem 3.2. Let A, P{z) andQA^z) as in Theorem \3.1\ We further 
assume P{z) is AA-nilpotent. Then, 

(a) QA,t{z) is the unique formal power series solution of the follow- 
ing Cauchy problem: 



(3.13) 

(6) For any k > 1, we have 



QA,t=o{z) = P{z). 



j.m—1 

(3.14) Qlt(^) = yT^ TT TTT^"^"^'' 

^ ^ ; 2™m!(m + A;)! ^ 



Applying the same strategy to Theorem 3.2 in jZ2] , we get the fol- 
lowing theorem. 

Theorem 3.3. Let A, P{z) and QA,t{z) as in Theorem \3.^ For any 
non-zero s E C, set 

k=0 

Then, Vt^s{z) is the unique formal power series solution of the following 
Cauchy problem of the heat-like equation: 



(3.15) 



'^{z) = iAAVtAz), 
Ut=o,s{z) = exp{sP{z)). 
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3.2. Some Criteria of A^-Nilpotency. In this subsection, witli tlie 
notation and remarks fixed in tlie previous subsection in mind, we 
apply the hnear automorphism $[/ to transform some criteria of Hessian 
nilpotency derived in [Z2] and jZ3] to criteria of A^-nilpotency {A G 
SM{n,C)) (see Proposition [331 Ell [321 and ElO] below). 

Proposition 3.4. Let A be given as in Eq. { \2.8^ . Then, for any P{z) G 
An, it is AA-nilpotent iff the submatrix of U'^{HesP)U consisting of 
the first r rows and r columns is nilpotent. 

In particular, when r = n, i.e. is full rank, any P{z) G 'is 
A^-nilpotent iff {HesP) U is nilpotent. 

Proof: Let P{z) be as in Eq. fl3.ll) . Then, as pointed earlier, P{z) 
is A^-nilpotent iff P{z) is A^-nilpotent. 

If r = n, then by Theorem 11.21 . P{z) is Ar-nilpotent iff HesP(2;) is 
nilpotent. But note that in general we have 

(3.16) ResPiz) = ResP{Uz) = U^[{I{es P){Uz)] U. 

Therefore, HesP(z) is nilpotent iff [{B.es P){Uz)]U is nilpotent iff, 
with z replaced by U~^z, If^ [{Res P){z)]U is nilpotent. Hence the 
proposition follows in this case. 

Assume r < n. We view Aj. as a subalgebra of the polynomial 
algebra K[zi, . . . , z^], where K is the rational field C(zr+i, ■ ■ ■ ,Zn)- By 
Theorem 1 1 . 2 1 and Lefschetz's principle, we know that P is A^-nilpotent 
iff the matrix \ jr-^) is nilpotent. 

Note that the matrix ( gf^j- ) is the submatrix of Hes P{z) con- 

sisting of the first r rows and r columns. By Eq. fl3.16p . it is also the sub- 
matrix of f/^[Hes P{U z)] U consisting of the first r rows and r columns. 

Replacing z by U^^z in the submatrix above, we see ( af^-j is 

nilpotent iff the submatrix of Wellies P{z)] U consisting of the first r 
rows and r columns is nilpotent. Hence the proposition follows. □ 



Note that, for any homogeneous quadratic polynomial P{z) = z'^Bz 
with B G SM{n,C), we have HesP(z) = 2B. Then, by Proposition 
13.41 we immediately have the following corollary. 

Corollary 3.5. For any homogeneous quadratic polynomial P{z) = 
z'^Bz with B G SM{n,'C), it is A^-nilpotent iff the submatrix of 
If^B U consisting of the first r rows and r columns is nilpotent. 
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Proposition 3.6. Let A be given as in Eq. ( \2.8i\ . Then, for any P{z) G 
An with o{P{z)) > 2, P{z) IS AA-mlpotent iff A^P™ = for any 
1 < m < r . 

Proof: Again, we let P{z) be as in Eq. ( 13. ip and note that P{z) is 
A^-nilpotent iff P{z) is Ar-nilpotent. 

Since r < n. We view Ar as a subalgebra of tlie polynomial algebra 
zi, . . . ,Zr], where K is the rational field C{zr+i, . . . , Zn). By Theorem 
11.21 and Lefschetz's principle (if r < n), we have P{z) is A^-nilpotent 
iff A^P"^ = for any 1 < m < r. On the other hand, by Eqs. (12.61) 
and (I22D, we have (^A^^P^j = A^P"' for any m > 1. Since is 

an automorphism of An, we have that, A™P™ = for any 1 < m < r 
iff A^P'" = for any 1 < m < r. Therefore, P{z) is A^i-nilpotent iff 
/\rnpm _ g ^^y- 1 < ^ < Hence the proposition follows. □ 

Proposition 3.7. For any A G SGL{n,C) and any homogeneous 
P{z) G An of degree d > 2, we have, P{z) is AA-nilpotent iff, for 
any (3 eC, {I3dY~'^P{z) is A-nilpotent, where (3d-= -D). 

Proof: Let A be given as in Eq. (12. 8 p and P{z) as in Eq. (13. ID . Note 
that, \l/^^(Ayi) = An (for A^ is of full rank), and P{z) is A^-nilpotent 
iff P{z) is A„-nilpotent. 

Since P is also homogeneous of degree d > 2, by Theorem 11.21 in 
[Z3] . we know that, P{z) is A„-nilpotent iff, for any /3 G C", /3^"^P is 
A„-nilpotent. Note that, from Lemma [2. H (6), we have 

= {Uf3,D) 
= {Uf3)D, 

and 

(3.17) ^uiPi'^P) = ^c/(/5d)'~'$i/(P) = {UP)i-^P 

Therefore, by Lemma [231 (a), Pi'^P is A„-nilpotent iff iUp)f^^P is 
A^-nilpotent since \l/[/(A„) = A^^. Combining all equivalences above, 
we have P{z) is A„-nilpotent iff, for any P G C", {Up^j^'^P is A-- 
nilpotent. Since U is invertible, when /3 runs over C" so does Up. 
Therefore the proposition follows. □ 

Let {ci \ 1 < i < n} he the standard basis of C". Applying the propo- 
sition above to /5 = Cj (1 < i < n), we have the following corollary. 
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Corollary 3.8. For any homogeneous A A-nilpotent polynomial P{z) G 
An of degree d>2, Df-^P{z) {1 < i < n) are also AA-nilpotent. 

We think that Proposition 13.71 and Corollary 13.81 are interesting be- 
cause, due to Corollary 13.51 it is much easier to decide whether a qua- 
dratic form is A^-nilpotent or not. 

To state the next criterion, we need fix the following notation. 

For any A G SGL{n, C), we let X^(C"') be the set of isotropic vectors 
M G C" with respect to the C-bilinear form (■, When yl = J„xn, we 
also denote Xa(C") simply by of X(C"). 

For any P G C", we set ha{z) := {a,z). Then, by applying $[/ to 
a well-known theorem on classical harmonic polynomials, which is the 
following theorem for A = J„xn (see, for example, |Hej and [Tj), we 
have the following result on homogeneous A^-nilpotent polynomials. 

Theorem 3.9. Let P be any homogeneous polynomial of degree d > 2 
such that AaP = 0. We have 

k 

(3.18) P{z) = Y,hi{^) 

1=1 

for some k >1 and ai G X^(C") (1 < « < fc). 

Next, for any homogeneous polynomial P{z) of degree ci > 2, we 
introduce the following matrices: 

(3-19) Sp:=((ai,aj)A)fcxfc. 

(3.20) np:=Ua,,a^)Aht\z)) . 

\ ■'J kxk 

Then, by applying $^ to Proposition 5.3 in [Z2j (the details will 
be omitted here), we have the following criterion of A^-nilpotency for 
homogeneous polynomials. 

Proposition 3.10. Let P{z) be as given in Eq. i\3.18^ . Then P{z) is 
Aji-nilpotent iff the matrix Qp is nilpotent. 

One simple remark on the criterion above is as follows. 
Let B be the kxk diagonal matrix with haX^) (1 < i < /c) as the 
z*'' diagonal entry. For any 1 < j < k, set 

(3.21) fip,:= B^EpB'-'-^ = {KM^^o.^)KT')- 

Then, by repeatedly applying the fact that, for any C,D E M{k, C), 
CD is nilpotent iff so is DC, it is easy to see that Proposition 13. 101 can 
also be re-stated as follows. 
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Corollary 3.11. Let P{z) he given by Eg. ([SAM with d > 2. Then, 
for any 1 < j < d — 2 and m > 1, P{z) is AA-nilpotent iff the matrix 
Qp-j is nilpotent. 

Note that, when d is even, we may choose j = {d — 2)/2. So P is 
A^-nilpotent iff the symmetric matrix 

(3.22) fip^(,_2)/2 = «,)^/.(^-^)/^) 

is nilpotent. 

3.3. Some Results on the Vanishing Conjecture of the 2nd 
Order Homogeneous Differential Operators with Constants 
Coefficients. In this subsection, we transform some known results of 
VC for the Laplace operators A„ (n > 1) to certain results on VC for 
{A e SGllnX))- 
First, by Wang's theorem [Waj . we know that JC holds for any 
polynomial maps F{z) with degF < 2. Hence, JC also holds for 
symmetric polynomials F{z) = z — VP{z) with P{z) e C[^] of degree 
d < 3. By the equivalence of JC and VC for the Laplace operators 
established in [Z2j . we know VC holds if A = A^ and P{z) is a HN 
polynomials of degree d < 3. Then, applying the linear automorphism 
$(7, we have the following proposition. 

Theorem 3.12. For any A G SGL{n, C) and AA-mlpotent P{z) G 
{not necessarily homogeneous) of degree d < 3, we have \^p^+'^ = Q 
when m >> 0, i.e. VC[?2] holds for A and P{z). 

Applying the classical homogeneous reduction on JC (see |BCW] . 
^) to associated symmetric maps, we know that, to show VC for A„ 
{n > 1), it will be enough to consider only homogeneous HN polyno- 
mials of degree 4. Therefore, by applying the linear automorphism $[/ 
of An, we have the same reduction for HVC too. 

Theorem 3.13. To study HVC in general, it will be enough to con- 
sider only homogeneous P{z) & A of degree 4. 

In |BE2] and |BE3] it has been shown that JC holds for symmetric 
maps F{z) = 2 — VP(z) {P{z) G An) if the number of variables n is less 
or equal to 4, or n = 5 and P{z) is homogeneous. By the equivalence of 
JC for symmetric polynomial maps and VC for the Laplace operators 
established in [Z2j . and Proposition 12.81 and Corollary 12.61 we have the 
following results on VC and HVC. 

Theorem 3.14. (a) For any n > 1, VC[?2] holds for any A G D2 0/ 
rank 1 < r < 4. 

(b) HVC [5] holds for any A G D2[5]. 
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Note that the following vanishing properties of HN formal power 
series have been proved in Theorem 6.2 in |Z2j for the Laplace operators 
> 1). By applying the linear automorphism one can show 
it also holds for any A-nilpotent (A G D2) formal power series. 

Theorem 3.15. Let A G (^iT'd P{z) G An he A-nilpotent with 

o{P) > 2. The following statements are equivalent. 

(1) A™P"^+^ = when m » 0. 

(2) There exists h > 1 such that A™'P™+''o = when m» 0. 

(3) For any fixed k > 1, A'^P™+^ = when m» 0. 

By applying the linear automorphism ${/, one can transform Theo- 
rem 1.5 in [EZJ on VC of the Laplace operators to the following result 
on VC of A G ©2- 

Theorem 3.16. Let A G I!>2[n] and P{z) G An any A-nilpotent poly- 
nomial with o{P) > 2. Then VC holds for A and P{z) iff, for any 
g{z) G An, we have A"^{g{z)P'^) = when m >> 0. 



In [EZj . the following theorem has also been proved for A = A 
Next we show it is also true in general. 



n- 



Theorem 3.17. Let A G SGL{n,C) and P{z) G An a homogeneous 
/S. A-nilpotent polynomial with degP > 2. Assume that a^-^i^z) : = 
z'^A'^z and the partial derivatives |^ (1 < « < "n.) have no non-zero 
common zeros. Then HVC[n] holds for Aa and P{z). 

In particular, if the projective subvariety determined by the ideal 
{P{z)) of An is regular, HVC[n] holds for A a andP{z). 

Proof: Let P as given in Eq. (13. ip . By Theorem 1.2 in |EZ] . we know 
that, when a2{z) := X]r=i partial derivatives |^ (1 < "i < 



n) 



have no non-zero common zeros, HVC[r;,] holds for A„ and P. Then, 
by Lemma [2.51 (6), HVC[n] also holds for A a and P. 

But, on the other hand, since U is invertible and, for any 1 < i < n, 

dP " dP 

el = Y.-^^d;.^uz)^ 

£72 (2;) and If- (1 < « < have no non-zero common zeros iff ct2{z) and 
^{Uz) {1 < i < n) have no non-zero common zeros, and iff, with z 
replaced by U~^z, a2{U~^z) = (7^-1(2) and ^{z) {I < i < n) have no 
non-zero common zeros. Therefore, the theorem holds. □ 
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3.4. The Vanishing Conjecture for Higher Order Differential 
Operators with Constant Coefficients. Even though the most in- 
teresting case of VC is for A G ©2; at least when JC is concerned, 
the case of VC for higher order differential operators with constant 
coefficients is also interesting and non-trivial. In this subsection, we 
mainly discuss VC for the differential operators (a G N"). At the 
end of this subsection, we also recall a result proved in |Z3j which says 
that, when the base field has characteristic p > 0, VC, even under a 
weaker condition, actually holds for any differential operator A (not 
necessarily with constant coefficients). 

Let 13 j G C" (1 < j < £) be linearly independent and set 5j := D). 
Let A = nj=i 5f with % > 1 (1 < j < £). 

When £ = 1, VC for A can be proved easily as follows. 

Proposition 3.18. Let 5 G Di[2;] and K = 5^ for some k > 1. Then 

(a) A polynomial P{z) is A-nilpotent if {and only if) AP = 0. 
(6) VC holds for A. 

Proof: Applying a change of variables, if necessary, we may assume 
(5 = Di and A = D^. 

Let P{z) G C[z] such that AP{z) = D'[P{z) = 0. Let d be the degree 
of P{z) in zi. From the equation above, we have k > d. Therefore, for 
any m > 1, we have km > dm which implies A^P{z)™' = D\^P"^{z) = 
0. Hence, we have (a). 

To show (6), let P{z) be a A-nilpotent polynomial. By the same 
notation and argument above, we have k > d. Choose a positive integer 
N > Then, for any m > N, we have m > which is equivalent 

to (m + l)d < km. Hence we have A"'P{z)"'+^ = D'l"'P"^+\z) = 0. 
□ 

In particular, when /c = 1 in the proposition above, we have the 
following corollary. 

Corollary 3.19. VC holds for any differential operator A G Di. 

Next we consider the case i > 2. Note that, when i = 2 and ai = 
02 = 1. A G ©2 and has rank 2. Then, by Theorem 13.141 we know VC 
holds for A. 

Besides the case above, VC for A = 11^=1 "with i > 2 seems to 
be non-trivial at all. Actually, we will show below, it is equivalent to a 
conjecture (see Conjecture I3.2ip on Laurent polynomials. 

First, by applying a change of variables, if necessary, we may (and 
will) assume A = D'^ with a G (N^)^. Secondly, note that, for any 
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b G N" and h{z) G Clz], D^h{z) = iff tlie lioiomorpliic part of tlie 
Laurent polynomial z~^h{z) is zero. 

Now we fix a P{z) G C[z\ and set f{z) := z~^P{z). With the obser- 
vation above, it is easy to see that, P{z) is D^-nilpotent iff the holo- 
morphic parts of the Laurent polynomials f"^{z) (m > 1) are all zero; 
and VC holds for A and P{z) iff the holomorphic part of P{z)f"^{z) 
is zero when m >> 0. Therefore, VC for D'^ can be restated as follows: 

Re-Stated VC for A = D'^ : Let P{z) G An and f{z) as above. 
Suppose that, for any m > 1, the holomorphic part of the Laurent poly- 
nomial f"^{z) is zero, then the holomorphic part of P{z)f^{z) equals 
to zero when m >> 0. 

Note that the re-stated VC above is very similar to the following 
non-trivial theorem which was first conjectured by O. Mathieu [Ma] 
and later proved by J. Duistermaat and W. van der Kallen [DKj . 

Theorem 3.20. Let f and g he Laurent polynomials in z. Assume that, 
for anym > 1, the constant term of f^ is zero. Then the constant term 
gjm gqy^Qig iq ^gj-Q when m » 0. 

Note that, Mathieu's conjecture |Maj is a conjecture on all real com- 
pact Lie groups G, which is also mainly motivated by JC. The the- 
orem above is the special case of Mathieu's conjecture when G the 
n-dimensional real torus. For other compact real Lie groups, Math- 
ieu's conjecture seems to be still wide open. 

Motivated by Theorem I3.20[ the above re-stated VC for A = D'^ 
and also the result on VC in Theorem 13.161 we would like to propose 
the following conjecture on Laurent polynomials. 

Conjecture 3.21. Let f and g be Laurent polynomials in z. Assume 
that, for any m > 1, the holomorphic part of is zero. Then the 
holomorphic part gf"^ equals to zero when m >> 0. 

Note that, a positive answer to the conjecture above will imply VC 
for A = (a G N") by simply choosing g{z) to be P{z). 

Finally let us to point out that, it is well-known that JC does not 
hold over fields of finite characteristic (see [BCW]. for example), but, 
by Proposition 5.3 in [Z3|, the situation for VC over fields of finite 
characteristic is dramatically different even though it is equivalent to 
JC over the complex field C 

Proposition 3.22. Let k be afield of char, p > and A any differential 
operator of k[z]. Let f G k[[z]]. Assume that, for any 1 < m < p — 1, 



22 



WENHUA ZHAO 



there exists Nm > such that A^™/™ = 0. Then, A™/™+i = when 
m » 0. 

From the proposition above, we immediately have the following corol- 
lary. 

Corollary 3.23. Let k be a field of char, p > 0. Then 
(a) VC holds for any differential operator A of k[z]. 
(6) // A strictly decreases the degree of polynomials. Then, for any 

polynomial f G k[z\ [not necessarily A-nilpotent) , we have f"^^^ = 

when m » 0. 

4. A Remark on A-Nilpotent Polynomials and Classical 
Orthogonal Polynomials 

In this section, we first in subsection 14.11 consider the "formal" con- 
nection between A-nilpotent polynomials or formal power series and 
classical orthogonal polynomials, which has been discussed in Section 
[1] (see pageH]). We then in subsection 14. 21 transform the isotropic prop- 
erties of homogeneous HN polynomials proved in [Z2j to isotropic prop- 
erties of homogeneous A^-nilpotent {A G SGL{n, C)) polynomials (see 
Theorem I4.1UI and Corollary 14.111) . Note that, as pointed in Section 
[H the isotropic results in subsection 14.21 can be understood as some 
natural consequences of the connection of A-nilpotent polynomials and 
classical orthogonal polynomials discussed in subsection 14. 1[ 

4.1. Some Classical Orthogonal Polynomials. First, let us recall 
the definition of classical orthogonal polynomials. Note that, to be 
consistent with the tradition for orthogonal polynomials, we will in 
this subsection use x = (xi, X2, . . . , x„) instead of z = {zi,Z2, . . . , Zn) 
to denote free commutative variables. 

Definition 4.1. Let B he an open set of and w{x) a real valued 
function defined over B such that w{x) > for any x E B and < 
J^w{x)dx < oo. A sequence of polynomials {fm{x) |m G N"} is said 
to be orthogonal over B if 

(1) deg/m = |m| for any m G N". 

(2) Jg fm{x)f]f_{x)w{x) dx = for any m 7^ k G N". 

The function w{x) is called the weight function. When the open 
set B C M" and w{x) are clear in the context, we simply call the 
polynomials fm{x) (m G N"') in the definition above orthogonal poly- 
nomials. If the orthogonal polynomials fm{x) (m G N") also satisfy 

f^{x)w{x)dx = 1 for any m G N"', we call fm{x) (m G N"') or- 
thonormal polynomials. Note that, in the one dimensional case w{x) 
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determines orthogonal polynomials over B up to multiplicative con- 
stants, i.e. if fm{x) {m > 0) are orthogonal polynomials as in Defi- 
nition U]T1 then, for any G (m > 0), amfm {fn > 0) are also 
orthogonal over B with respect to the weight function w{x). 

The most natural way to construct orthogonal or orthonormal se- 
quences is: first to list all monomials in an order such that the degrees 
of monomials are non-decreasing; and then to apply Gram-Schmidt 
procedure to orthogonalize or orthonormalize the sequence of mono- 
mials. But, surprisingly, most of classical orthogonal polynomials can 
also be obtained by the so-called Rodrigues' formulas. 

We first consider orthogonal polynomials in one variable. 

Rodrigues' formula: Let fm{x) {m > 0) be the orthogonal polyno- 
mials as in Definition \4-i\ Then, there exist a function g{x) defined 



over B and non-zero constants Cm {rn > 0) such that 
(4.1) Ux) = c^nj{x)-'^{w{x)g"^{x)). 



Let P{x) := g{x) and A:= w{x)~^ (^) w{x), where, throughout this 
paper any polynomial or function appearing in a (differential) operator 
always means the multiplication operator by the polynomial or function 
itself. Then, by Rodrigues' formula above, we see that the orthogonal 
polynomials {fm{x) | m > 0} have the form 

(4.2) /„(x)=c^A'"P"^(x), 

for any m > 0. 

In other words, all orthogonal polynomials in one variable, up to 
multiplicative constants, has the form jA^P™ | m > 0} for a single 
differential operator A and a single function P{x). 

Next we consider some of the most well-known classical orthonor- 
mal polynomials in one variable. For more details on these orthogonal 
polynomials, see [Si], [AS] , |DXj . 

Example 4.2. (Hermite Polynomials) 

(a) P = M and the weight function w{x) = e~^ . 

(b) Rodrigues ' formula: 

Hm{x) = {-ire^\^re-^\ 
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(c) Differential operator A and polynomial P(x): 
P{x) = 1, 

{d) Hermite polynomials in terms of A and P{x): 
H^{x)^{-irA^P'^{x). 

Example 4.3. (Laguerre Polynomials) 

(a) 5 = ]R+ and w{x) = x"e~^ {a > -1). 
(6) Rodrigues ' formula: 

(c) Differential operator A and polynomial P{x): 

A„ = x-^e^{£){e-^x^) = £ + {ax-^ - 1), 
P{x) = X, 

{d) Laguerre polynomials in terms of A and P{x): 

Lrr,{x)^—A^P^{x). 

ml 

Example 4.4. (Jacobi Polynomials) 

(a) B = (-1, 1) and w{x) = (1 - + xf, where a,P> -1. 

(b) Rodrigues ' formula: 

p^^ix) = ^ — '—(1 - + x)-^(-^y'\i - + x)''+'' 

(c) Differential operator A and polynomial P{x): 

A = (1 - + - + 

^±-a{l-x)-'+P{l + x)-\ 
dx 

and 

P{x) = 1 - 

(d) Laguerre polynomials in terms of A and P{x): 



PZ'Hx) = ^ '—A'^p-^ix) 



2™m! 

A very important special family of Jacobi polynomials are the Gegen- 
bauer polynomials which are obtained by setting a = (3 = X — 1/2 for 
some A > —1/2. Gegenbauer polynomials are also called ultraspherical 
polynomials in the literature. 
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Example 4.5. (Gegenbauer Polynomials) 

(a) B = (-1, 1) and w{x) = (1 - x^^-^^^, where X > -1/2. 
(6) Rodrigues ' formula: 

pA/ N _ _ ^2Nl/2-A/_^Nm/i _ 2^m+A-l/2 

where, for any c G M and A; G N, (c)fc = c(c + 1) ■ ■ ■ (c + A; — 1). 
(c) Differential operator K and polynomial P{x): 



d 



2\A-l/2 



(4.3) A = (l-x2)i/2-^(^)(l-x 

ax 

_ (2A- l)x 

(ix (1 — x^) ' 

anc? 

P(x) = 1 - x^. 
(d) Laguerre polynomials in terms of A and P{x): 

^ 2-(A + l/2)^ ^ ^ 

Note that, for the special cases with A = 0, 1, 1/2, the Gegenbauer 
Polynomials -P^(^) called the Chebyshev polynomial of the first 
kind, the second kind and Legendre polynomials, respectively. Hence 
all these classical orthogonal polynomials also have the form of A'^P'" 
(m > 0) up to some scalar multiple constants with P{x) = 1 — x^ 
and the corresponding special forms of the differential operator A in 
Eq. gJl). 

Remark 4.6. Actually, the Gegenbauer polynomials are more closely 
and directly related with VC in some different ways. See |Z4j for more 
discussions on connections of the Gegenbauer polynomials with VC. 

Next, we consider some classical orthogonal polynomials in several 
variables. We will see that they can also be obtained from certain 
sequences of the form {A^P"* | m > 0} in a slightly modified way. One 
remark is that, unlike the one- variable case, orthogonal polynomials 
in several variables up to multiplicative constants are not uniquely 
determined by weight functions. 

The first family of classical orthogonal polynomials in several vari- 
ables can be constructed by taking Cartesian products of orthogonal 
polynomials in one variable as follows. 
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Suppose {/m I > 0} is a sequence of orthogonal polynomials in one 
variable, say as given in Definition I4.1[ We fix any n >2 and set 

n 

(4.4) Vr(a;):=JJ w{xi), 

1=1 

n 

(4.5) Ux)■.= l[fmXx^), 

1=1 

for any x G -B^" and m G N". 

Then it is easy to see that the sequence {/m(a;) | m G N*^} are orthog- 
onal polynomials over 5^" with respect to the weight function W{x) 
defined above. 

Note that, by applying the construction above to the classical one- 
variable orthogonal polynomials discussed in the previous examples, 
one gets the classical multiple Hermite Polynomials, multiple Laguerre 
polynomials, multiple J acohi polynomials audi, multiple G eg enhauer poly- 
nomials, respectively. 

To see that the multi-variable orthogonal polynomials constructed 
above can be obtained from a sequence of the form {A™P™(x) | m > 
0}, we suppose fm ijn > 0) have Rodrigues' formula Eq. (14.11) . Let 
s = (si, . . . , Sn) be n central formal parameters and set 

(4.6) A. ■.=W{x)-' ^(^)' 

n 

(4.7) P{x):=\[g{x;). 

i=l 

Let V^{x) (m G W) be the coefficient of s'" in Ai'"'pl'"l(x). Then, 
from Eqs. 04.11) . fl01) - (C71) . it is easy to check that, for any m G N", 
we have 

m! 

(4.8) fm{x) = Cm^r V^{x), 

|m|! 

where = nr=i ^m,. 

Therefore, we see that any multi-variable orthogonal polynomials 
constructed as above from Cartesian products of one- variable orthogo- 
nal polynomials can also be obtained from a single differential operator 
A^ and a single function P{x) via the sequence {A^P*" | m > 0}. 

Remark 4.7. Note that, one can also take Cartesian products of dif- 
ferent kinds of one-variable orthogonal polynomials to create more or- 
thogonal polynomials in several variables. By a similar argument as 
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above, we see that all these multi-variable orthogonal polynomials can 
also be obtained similarly from a single sequence {A^P"^ | m > 0}. 

Next, we consider the following two examples of classical multi- 
variable orthogonal polynomials which are not Cartesian products of 
one-variable orthogonal polynomials. 

Example 4.8. (Classical Orthogonal Polynomials over Unit 
Balls) 

(a) Choose B to be the open unit ball B" of M" and the weight func- 
tion 

W,{x) = {l-\\x\\^Y~'/\ 

where \ \x\ \ = ^"^^a;^ and fi > 1/2. 

(6) Rodrigues' formula: For any m G N", set 

U ixY g ||dP)i-i+'-^/^ 

2Hm!(/i+l/2)|^|9xr---9a;-"^ ""^ 

Then, by Proposition 2.2.5 in [DXj . {f/m(a^) | m G N"} are orthonormal 
over B" with respect to the weight function W^j_{x). 

(c) Differential operator Kg and polynomial P {x) : Let s = (si, . . . , s„) 
be n central formal parameters and set 



. dxi 



P(x) :=1 — I |a;| 



Let Vm{x) (m G N") be the coefficient of s"" m Ai"''pl™l(x). Then 
from the Rodrigues type formula above, we have, for any m G N"", 

(-l)H(2/i)|^l 
^ ^ 2l'"l|m|!(/i+ l/2)|„i| ^ ' 

Therefore, the classical orthonormal polynomials {Umix) | m G N"} 
owerB" can be obtained from a single differential operator Ag and P{x) 
via the sequence {A^P"* | m > 0}. 

Example 4.9. (Classical Orthogonal Polynomials over Sim- 
plices) 

(a) Choose B to be the simplex 

n 

T" = {x G M" I < 1; Xi,...,Xn > 0} 

i=l 



28 



WENHUA ZHAO 



in M" and the weight function 

(4.9) W^{x) =x^^---x;;"(l-|x|i)""+^~^/^ 

where > — 1/2 (1 < z < n + 1) and \x\i = J2^=i ^i- 

(b) Rodrigues' formula: For any m G N", set 

Then, {Um{x) \ m G N"} are orthonormal over T" with respect to the 
weight function Wk{x). See Section 2.3.3 of [DXJ for a proof of this 
claim. 

(c) Differential operator A and polynomial P{x): Let s = (si, . . . , s„) 
be n central formal parameters and set 




P{x) :=1 — 

Let 1^111(2;) (m £ N") 6e t/ie coefficient of s"^ in A^J^^ P^"^^ (x) . Then 
from the Rodrigues type formula in (b), we have, for any m G N", 

m! 

Um{x) = rrV^ix). 

|m|! 

Therefore, the classical orthonormal polynomials {Umix) | m G N"} 
over T" can be obtained from a single differential operator Ag and a 
function P{x) via the sequence {A^P™ | m > 0}. 

4.2. The Isotropic Property of A^-Nilpotent Polynomials. As 

discussed in Section [T], the "formal" connection of A-nilpotent polyno- 
mials with classical orthogonal polynomials predicts that A-nilpotent 
polynomials should be isotropic with respect to a certain C-bilinear 
form of An- In this subsection, we show that, for differential operators 
A = {A G SGL{n, C)), this is indeed the case for any homogeneous 
A-nilpotent polynomials (see Theorem 14. 101 and Corollaries I4.11ll4.12p . 

We fix any n > 1 and let z and D denote the n-tuples (zi, . . . , z„) 
and (Di, D2, . . . , Dn), respectively. Let A G SGL{n, C) and define the 
C-bilinear map 

(4.10) xyi„^ An 

if, g) f{AD)g{z), 
Furthermore, we also define a C-bilinear form 

(4.11) {■,-)A:An^An^ C 
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if, g) ^{f,g}Jz=o, 

It is straightforward to check that the C-bihnear form defined above 
is symmetric and its restriction on the subspace of homogeneous poly- 
nomials of any fixed degree is non-singular. Note also that, for any 
homogeneous polynomials f,gE An of the same degree, we have 

{f,g}A = {f,g)A- 

The main result of this subsection is the following theorem. 

Theorem 4.10. Let A G SGL{n,C) and P{z) G An a homogeneous 
AA-nilpotent polynomial of degree d > 3. Let be the ideal of An 
generated by a^-'^iz) := z'^A~'^z and ^ {1 < i < n). Then, for any 
f{z) G J(-P) and m >0, we have 

(4.12) {/, a;jp™+i}a = f{AD) a;jp'"+^ = 0. 



Note that, by Theorem 6.3 in |Z2j . we know that the theorem does 
hold when A = In and = An- 

Proof: Note first that, elements of An satisfying Eq. (14.121) do form 
an ideal. Therefore, it will be enough to show 17^-1(2;) and (1 < 
i < n) satisfy Eq. (14.121) . But Eq. (I4.12p for 0-^-1(2) simply follows the 
facts that aA-^{Az) = z'^Az and (7a-i{AD) = Aa- 

Secondly, by Lemma [2!2| we can write A = UU"^ for some U = {uij) G 
GL(ra,C).^Then, by Eq. ([52D, we have *t/(An) = or ^^^(A^) = 
A„. Let P{z) := <i>u\P) = P{Uz). Then by Lemma [531 («), ^ is a 
homogeneous A„-nilpotent polynomial, and by Eq. (12.61) . we also have 

(4.13) <i>^i(a;jp"^+^) = A^P™+^ 

By Theorem 6.3 in [Z2] . for any 1 <i <n and m > 0, we have. 



dP 

dzi 



(D) (^A™P™+i) = 



Since 



we further have, 



dP " dP 

* k=l 



j^^M^iUD) (a™P'"+i)=0. 

k=l 

Since U is invertible, for any 1 < i < ra, we have 

BP / ~ \ 

(4.14) d7^^^^ (^A'^P"'+^j = 0. 
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Combining the equation above with Eq. f l4.13p . we get 

BP 

— {UD)%^ (A:JP'"+i) = 0. 
dP 

BP 

(4.15) ($^_(f/Z))$^i) (A™P™+i) = 0. 

By Lemma EH (6), Eq. (HTTHl) and the fact that A = UW , we get 

dP BP 

— {UU^D) (A:JP'"+i) = j^iAD) (A:^P'"+i) = 0, 

which is Eq. fHTT^ for (1 < z < n). □ 

Corollary 4.11. Let A he as in Theorem 4-10 and P{z) he a homoge- 



neous AA-nilpotent polynomial of degree d>d>. Then, for any m> 1, 
^mpm+i isotropic with respect to the C-hilinear form (-, ■)a, i-c. 

(4.16) (A™P™+\ A:JP"^+^)a = 0. 

In particular, we have (P, P)a = 0. 

Proof: By the definition Eq. (14.111) of the C-bilinear form (■, ■)a and 
Theorem 14.101 it will be enough to show that P and A^P™""*"^ (m > 1) 
belong to the ideal generated by the polynomials |^ (1 < < ■«-) (here 
we do not need to consider the polynomial cr^-i (z)). But this statement 
has been proved in the proof of Corollary 6.7 in |Z2j . So we refer the 
reader to |Z2j for a proof of the statement above. □ 

Theorem 14.101 and Corollary 14.111 do not hold for homogeneous HN 
polynomials P{z) of degree d = 2. But, by applying similar arguments 
as in the proof of Theorem 14.101 above to Proposition 6.8 in |Z2J, one 
can show that the following proposition holds. 



Proposition 4.12. Let A he as in Theorem\4.1(J\ and P{z) a homoge- 



neous Aj^-nilpotent polynomial of degree d = 2. Let 3{P) the ideal of 
C[z] generated by P{z) and (7^-1(2;). Then, for any f{z) G 3{P) and 
m > 0, we have 



(4.17) {/, A^^P'^+ijA = f{AD) A^^P^+i = 



In particular, we still have (P, P)a = 0. 
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